In some conventional methods of correcting blur due to camera shake, the amount of camera blur between frames is estimated by using a technique of estimating a motion vector using two pictures so that the blur due to camera shake can be corrected. A Moving Picture Experts Group (MPEG) technique is representative of such a technique.
In this method, a picture is divided into rectangular regions, and the amount of motion between frames is calculated for each of the regions. The amount of motion of the whole picture is estimated from motion vectors of the respective regions so that the picture can be finally corrected. Such a method performed using motion vectors has problems in accuracy and computational cost because of the nature of algorithm. This limits the magnitude of maximum detectable blur due to camera shake. There is a trade-off between the magnitude of maximum detectable blur due to camera shake and the computational cost. The larger the magnitude of maximum detectable blur due to camera shake, the higher the computational cost. Thus, the magnitude of maximum detectable blur due to camera shake is usually determined based on assumed magnitude of blur due to camera shake. In order to detect large blur due to camera shake using the method, a range of detectable blur due to camera shake needs to be wide. On the other hand, the range coincides with a solution space. That is, the wider the range is, the more probable it is that an obtained solution results in a local solution. In this case, blurs are not detected with sufficient accuracy, and the magnitude of blurs in pictures taken during walking or without using a finder cannot be supported.
On the other hand, there is a method of correcting blur which is too large to correct using motion vectors. In this method, blur is corrected based on feature point matching. Unlike the method using motion vectors which are information of respective regions, the method is performed using several points on objects which are present in both two pictures taken consecutively. Among the points, the one which is in both of the two pictures and detectable by picture processing is referred to as a feature point. The feature-point-based matching is a method in which a motion between two pictures is estimated by matching feature points between frames. A rotation matrix representing the amount of blur can be estimated by the matching, and the blur is corrected using the rotation matrix.
In the feature-point-based matching, the magnitude of blur due to camera shake is usually unknown and no information on the object in the picture is available. It is therefore impossible to determine in advance which of the feature points can be used to make an accurate estimation of blur due to camera shake.
An appropriate combination of feature points is determined by an exhaustive search using a method such as a RANdom SAmple Consensus (RANSAC). Then, the amount of blur is estimated usually using a pair of feature points (inliers) determined as the most suitable combination by a preset evaluation function.
In such feature-point-based matching, feature points are matched based on similarity between two frames. The size of a solution space therefore depends on the number of the feature points. Accordingly, this method does not require a high computational cost and the probability that an obtained solution results in a local solution is low compared to the method in which information on respective regions, that is, a motion vector is used, even when a range of detectable blur due to camera shake is wide.
Therefore, feature-point-based matching allows estimation of large blur due to camera shake in pictures taken during walking or without using a finder.
However, in the feature-point-based matching, a feature point to be used for estimation of the amount of blur needs to be the one obtained from a distant view region of a picture. This is a problem equivalent to the problem what is used as a reference for correcting blur due to camera shake.
In other words, a camera moves not with respect to an object but with respect to a distant view (background). It is therefore preferable to correct blur due to camera shake with reference to the distant view. It is for this reason that a feature point to be used is preferably the one in a distant view region.
The point is that blur remains in a picture corrected based on matching with reference to a close object such as a passerby, but the picture looks as if there was not the blur due to camera shake when matching is performed with reference to a distant view.
It should be noted that when a picture is divided into a near view region and a distant view region, the distant view region is a region showing an object relatively distant from the camera. For example, in a picture showing objects such as a person and a building or nature (trees, for example), the region showing the image of the person is a near view region, and the region showing the image of the building or the nature.